Landmark-based location of users

ABSTRACT

Humans identify location and directions based on visual cues, such as landmarks, rather than a precise coordinate grid. A database of landmarks can be used to determine the location of a user who can merely describe their location in the same manner they would to any human contact. The location of the user can be determined based on the landmarks described in the input, and the areas from which all of those landmarks are visible. Various databases can be used, including text-based and three-dimensional. Similarly, various mechanisms can be used to determine the user&#39;s location, including hierarchical approaches, query optimization approaches, and visibility percentage approaches. Ultimately, the user&#39;s location can be determined without the need for expensive, dedicated infrastructure, without compromising the user&#39;s security, and without subjecting the user to system outages.

TECHNICAL FIELD

[0001] This invention relates generally to locating a mobile user and,more particularly, relates to locating a user through visualobservations made by the user.

BACKGROUND OF THE INVENTION

[0002] An increasingly popular use of computing devices is to allow auser the ability to interact with their physical surroundings. Forexample, handheld computers can provide the user with information ontopics of interest near the user, such as restaurants, copy centers, orautomated teller machines. Similarly, laptop computers seek to allow auser to control devices located in close proximity to that laptop, suchas a printer located in the same room. Additionally, specializedcomputing devices, such as navigational systems found in manyautomobiles seek to provide the user with directions based on the user'scurrent location. A necessary element of all of these applications isthe ability to determine the location of the user.

[0003] Current systems for locating a user rely on extensive, andexpensive, infrastructure to support the system. One well known systemfor determining location is the Global Positioning System (GPS). GPS isa satellite based system in which the receiver, carried by the user,requires a signal from at least four satellites to determine thelocation of the user. A similar system, known as E-911, relies onsignals from a user's cellular telephone to triangulate the user'sposition. Two methods currently relied on by the e-911 system use eitherthe angle of the arrival of the user's signal, the time of arrival, orthe time difference of arrival. Both the GPS system and the E-911 systemrequire the installation and maintenance of expensive and dedicatedinfrastructure. GPS, for example, requires maintaining many satellites,and the E-911 system requires either the installation of directionalantennas on the cell phone and base station, or specialized equipment atthe base stations to provide time synchronization. In addition, theE-911 system may require the user to purchase a new cell phone. Thesesystems also suffer from other drawbacks inherent in their design. Forexample, the GPS system does not work indoors or in locations, such asdowntown areas, with significant obstructions to the signal pathsbetween the user and the satellites and cellular telephone basestations. Similarly, the E-911 system requires that the cell phone beable to communicate with multiple base stations simultaneously.

[0004] Additional location determining systems can be used in confinedareas, but they too suffer from the need to purchase and maintaindedicated infrastructure. One known system for determining the locationof a user indoors is the Active Badge system, that uses infrared beaconsand sensors located throughout an area to pinpoint the user's location.Additional systems rely on wireless communication between a transmitterand detectors placed throughout the indoor environment. Such systemssuffer from the same complex infrastructure problem as GPS or E-911. Ineither case, dedicated detection equipment must be installed throughoutthe indoor area. Because of the amount of infrastructure needed for evensmall areas, such systems cannot be scaled well to larger areas, such asa whole town or metropolitan area. Additionally, some of these systems,such as infrared based systems, can be impacted by environmentalconcerns, such as direct sunlight.

[0005] In addition to the above problems, some of the known systems fordetermining a user's location suffer from privacy concerns.Specifically, the user's position is determined by infrastructure thatis exclusively controlled by a central authority. Thus, the centralauthority could use the system to determine the location of a user evenwhen that user did not wish to learn their location, or did not wish tobe located. Furthermore, because all of these systems rely on a centralarchitecture, they all leave the user vulnerable to system outages whenthe infrastructure required is not properly operated or maintained bythe central authority. Often, the user is simply forced to rely on humaninteraction to determine their location.

SUMMARY OF THE INVENTION

[0006] Accordingly, the present invention is directed to identifying thelocation of a user without relying on dedicated infrastructure and acentral authority.

[0007] Additionally, the present invention is dedicated to identifyingthe location of a user based on landmarks or other visual cues visibleto the user from their current position.

[0008] The present invention is likewise directed to providing a userwith an accessible and efficient database of landmarks and other visualcues from which the user's position can be determined.

[0009] For centuries, people have guided one another by landmarks andvisual cues. For example, people will often identify intersections, notby the street names, but by the names of gas stations or stores locatedon that comer. Similarly, people often describe their position asrelative to specific landmarks, rather than using a more accuratecoordinate system. The present invention seeks to capitalize on thisinteraction to provide a more intuitive system for determining thelocation of a user, that does not require significant infrastructure,comprise the user's privacy, or leave the user vulnerable to system-wideoutages.

[0010] The present invention contemplates a database of landmarks in agiven area, such as a metropolis or a shopping mall. The database can betextual, associating landmarks with descriptive terms commonly used todescribe that landmark. Alternatively, or in addition, athree-dimensional topographical database of the environment can be used,which can provide information regarding the visibility of the landmarkfrom various locations, as well as distinctive features that can be usedto further specify the user's location.

[0011] Because of the nature of location determination, the presentinvention anticipates a portable computing device to interface with thedatabase and calculate the user's position. While a textual database canbe of a sufficiently small size to be wholly stored on the portablecomputing device, a three-dimensional topographical database may containtoo much information to be stored, in its entirety, on the portablecomputing device. Various techniques can be used to reduce the size ofthe database to more easily accommodate the generally smaller storagecapacities of storage devices used by portable computing devices. Forexample, the three-dimensional topographical database can be simplifiedby removing information the user is not likely to comment upon, such asthe texture of a wall of the building, or the windows of the building.Instead, the model can be downgraded such that it consists merely ofblock shapes that provide a minimum of information while continuing toallow the computation of the visibility regions. Alternatively, asubsection of the three-dimensional topographical model can bedownloaded to the portable computing device, and the remainder can beobtained from a server that has increased storage capacity and can storethe whole model. The portable computing device can request specificinformation from the server, or it can merely listen to broadcast ormulticast transmissions from the server providing additional informationfor a specific area or region. Similarly, a minimalist model, containingonly the block shapes, can be used in conjunction with servercommunication, such that the server can be contacted to provide thefeatures and elements, as needed, that were removed from the minimalistmodel.

[0012] With a portable computing device having access to a database oflandmark information, the user can provide input that will enable thesystem to determine the user's location. The portable computing devicecan be equipped with a pen-based or keyboard-based input device toaccept computer-readable information from the user, or the device can beequipped with a microphone and speech interpreting software to acceptvocal input. Using the appropriate input device, the user can describe,in varying degrees of specificity, what the user sees. Based on thedescription of the landmarks provided by the user, the portablecomputing device can access the database to determine the user'slocation. If insufficient information was provided, or the informationresults in the determination of more than one unique location, the usercan be prompted for more information. For example, the user can beprompted for specific information needed to resolve an ambiguity, or theuser can be prompted for more information, generally, about theirsurroundings.

[0013] Once the portable computing device has been provided with theuser's input, it can compare the user's descriptions of theirsurroundings to the database to identify the landmarks that the user isdescribing. Visibility regions, indicating those areas from which thelandmark can be seen, can be calculated for each landmark identified, orfor a more focused subset, such as those landmarks that have smallvisibility regions. If the intersection of the visibility regions yieldsonly one location, then the system can return it as the user's location.Alternatively, if the intersection of the visibility regions yields anarea, or multiple areas with differing probabilities of likelihood, thesystem can prompt the user for more information. The user's locationcan, therefore, be located by determining a location which uniquelymatches the description provided by the user of their currentsurroundings.

[0014] Additional features and advantages of the invention will be madeapparent from the following detailed description of illustrativeembodiments that proceeds with reference to the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015] While the appended claims set forth the features of the presentinvention with particularity, the invention, together with its objectsand advantages, may be best understood from the following detaileddescription taken in conjunction with the accompanying drawings ofwhich:

[0016]FIG. 1 is a block diagram generally illustrating an exemplarycomputer system on which the present invention resides;

[0017]FIG. 2 is a three-dimensional representation of an exemplary areain which the present invention can be used;

[0018]FIG. 3 is a two-dimensional representation of an exemplary area inwhich the present invention can be used; and

[0019]FIG. 4 is a flowchart illustrating one preferred embodiment of thepresent invention.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

[0020] The present invention contemplates the determination of thelocation of a user based on the user's description of theirsurroundings. A portable computing device can be used to accept userinput and determine the user's location, or communicate with additionalcomputing devices to determine the user's location. A database oflandmarks, and other visual cues, can be accessed to identify thelandmarks described by the user. The database can further be used todetermine the user's location by identifying a unique location thatmatches the user provided description of the user's view.

[0021] Turning to the drawings, wherein like reference numerals refer tolike elements, the invention is described hereinafter in the context ofa computing environment. Although it is not required for practicing theinvention, the invention is described as it is implemented bycomputer-executable instructions, such as program modules, that areexecuted by a computing device. Generally, program modules includeroutines, programs, objects, components, data structures and the likethat perform particular tasks or implement particular abstract datatypes.

[0022] The invention may be implemented in a variety of computer systemconfigurations, including stand-alone configurations, client-serverconfigurations, and client-infrastructure configurations. For example,the invention may be realized in portable computing devices, such ashand-held devices, laptop computers, pocket computers, wearablecomputers, dedicated electronic devices, or programmable consumerdevices. Alternatively, the invention may be realized in a client-serverconfiguration, including a client computing device, such as a personalcomputer, a micro-processor based specific computing device, or aportable computing device, as described above, and a server computingdevice, such as a personal computer, a server computer, multi-processorcomputer system, a minicomputer, or a mainframe computer. Furthermore,the invention may also be realized in a client-infrastructureconfiguration, including a client computing device such as thosedescribed above, and infrastructure computing devices, such as dedicatedbroadcast equipment, kiosks, network access points, or base stations.The invention may be practiced in distributed computing environments,such as the client-server environment and the client-infrastructureenvironment, described above, where tasks are performed by remoteprocessing devices that are linked through a communications network. Ina distributed computing environment, program modules may be located inboth local and remote memory storage devices.

[0023] Although the invention may be incorporated into many types ofcomputing environments as suggested above, the following detaileddescription of the invention is set forth in the context of an exemplarygeneral-purpose portable computing device 20.

[0024] Before describing the invention in detail, the computingenvironment in which the invention operates is described in connectionwith FIG. 1.

[0025] The portable computing device 20 includes a processing unit 21, asystem memory 22, and a system bus 23 that couples various systemcomponents including the system memory to the processing unit 21. Thesystem bus 23 may be any of several types of bus structures including amemory bus or memory controller, a peripheral bus, and a local bus usingany of a variety of bus architectures. The system memory includes readonly memory (ROM) 24 and random access memory (RAM) 25. A basicinput/output system (BIOS) 26, containing the basic routines that helpto transfer information between elements within the portable computingdevice 20, such as during start-up, is stored in ROM 24.

[0026] The portable computing device 20 can also include memory storagedevices, such as a hard disk drive 27 for reading from and writing to ahard disk 60, or an optical disk drive 30 for reading from or writing toa removable optical disk 31. Alternatively either the hard disk drive27, and corresponding hard disk 60, or the optical disk drive 30 andcorresponding optical disk 31 may be replaced with solid state memorystorage devices, including a solid state memory storage drive and memorycards, such as a Compact Flash (CF) memory card, a MultiMedia Card(MMC), or a SmartMedia (SM) card. The hard disk drive 27 and opticaldisk drive 30 are connected to the system bus 23 by a hard disk driveinterface 32 and an optical disk drive interface 34, respectively.Similarly, a solid state memory storage drive interface could be used toconnect the solid state memory storage drive to the system bus 23. Thememory storage devices provide nonvolatile storage of computer readableinstructions, data structures, program modules and other data for theportable computing device 20. It will be appreciated by those skilled inthe art that, while some types of computer readable media areparticularly called out in the description of a preferred embodiment,other types of computer readable media which can store data that isaccessible by a computer, such as magnetic cassettes, digital videodisks, Bernoulli cartridges, random access memories, read only memories,and the like may also be used in the exemplary operating environment.

[0027] A number of program modules may be stored on the hard disk 60,magnetic disk 29, optical disk 31, ROM 24 or RAM 25, including anoperating system 35, one or more applications programs 36, other programmodules 37, and program data 38. A user may enter commands andinformation into the portable computing device 20 through input devicessuch as a keyboard 40 and a microphone 42. Other input devices (notshown) may include a mouse, joystick, game pad, satellite dish, scanner,or the like. These and other input devices are often connected to theprocessing unit 21 through a serial port interface 46 that is coupled tothe system bus, but may be connected by other interfaces, such as aparallel port, game port or a universal serial bus (USB). The portablecomputing device 20 can also contain a visual output device, such as aLiquid Crystal Display (LCD) or other portable display technology, or anaudio output device, such as a speaker or headphones.

[0028] The portable computing device 20 may operate in a networkedenvironment using logical connections to one or more remote computers,such as a server computing device 49. The server computing device 49 maybe a server computer, a router, a minicomputer, a microcomputer, amulti-processor computing device, a network PC, a peer device or othercommon network node, and typically includes many or all of the elementsdescribed above relative to the portable computing device 20, althoughonly a memory storage device 50 has been illustrated in FIG. 1.Alternatively, or in addition, the portable computing device 20 can alsoutilize an infrastructure device 62, specifically designed to assist theportable computing device in determining the location of the user. Forexample, infrastructure device 62 can be a kiosk that can download adatabase containing information about landmarks in the vicinity of thekiosk. Alternatively, the infrastructure device 62 can be a wirelessnetwork access point, or another portable computing device that canuniquely identify its position to the portable computing device 20. Theinfrastructure device 63 is shown with an attached memory storage device63, though the infrastructure device 62 can access the memory storagedevice 63 through a variety of connections in addition to the directconnection shown, such as a wireless connection, or a networkconnection.

[0029] The logical connections depicted in FIG. 1 include a local areanetwork (LAN) 51 and a wireless network 52. Such networking environmentsare commonplace in offices, enterprise-wide computer networks, intranetsand the Internet. Additionally, such networking environments can rely onvarious protocols including Ethernet protocols, wireless Ethernetprotocols, Infra-Red Data Association (IrDA) protocols, Bluetoothprotocols, and the like. When used in a LAN networking environment, theportable computing device 20 is connected to the local network 51through a communication device such as a network interface or adapter53. When used in a wireless networking environment, the portablecomputing device 20 typically includes other communication devices, suchas a wireless communication device 61, connected to the system bus 23via the serial port interface 46. In a networked environment, programmodules depicted relative to the portable computing device 20, orportions thereof, may be stored in the remote memory storage device. Itwill be appreciated that the network connections shown are exemplary andother means of establishing a communications link between the computersmay be used.

[0030] In the description that follows, the invention will be describedwith reference to acts and symbolic representations of operations thatare performed by one or more computing devices, unless indicatedotherwise. As such, it will be understood that such acts and operations,which are at times referred to as being computer-executed, include themanipulation by the processing unit of the computer of electricalsignals representing data in a structured form. This manipulationtransforms the data or maintains it at locations in the memory system ofthe computer, which reconfigures or otherwise alters the operation ofthe computer in a manner well understood by those skilled in the art.The data structures where data is maintained are physical locations ofthe memory that have particular properties defined by the format of thedata. However, while the invention is being described in the foregoingcontext, it is not meant to be limiting as those of skill in the artwill appreciate that various of the acts and operations describedhereinafter may also be implemented in hardware.

[0031] In accordance with one aspect of the invention, a databasecontaining information about landmarks and visual cues is provided todetermine a user's location based on landmark identification inputprovided by the user. The database can be a simple textual databasecorrelating landmarks and descriptive terms commonly used to describethe landmark. Alternatively, or in addition, a three-dimensionaltopographical database can provide further information, includingvisibility regions, and noticeable physical traits. To minimize theburden on the user, the present invention contemplates the use of aportable computing device to identify the user's location. Thethree-dimensional topographical database, together with additionalsmaller databases, such as the simple textual database, can be stored onthe portable computing device by removing some information from thethree-dimensional topographical database, or providing only a portion ofthe topographical database, with the remainder to be downloaded asneeded.

[0032] In accordance with another aspect of the invention, the user'slocation can be identified by reference to input received from the userdescribing nearby landmarks and other visual cues. Based on the user'sinput, the portable computing device can reference the database storedtherein, or can reference an external database, and can determine thelandmarks described by the user. Based on the locations of thelandmarks, and their visibility regions, the portable computing devicecan determine the user's location. If the determined landmarks arevisible from more than a single location, or the probability that theuser is in a particular location remains too low, the user can beprompted for additional information, such as more specific informationabout landmarks the user has already described, or information about anadditional landmark. This additional information can then be used toidentify the location of the user.

[0033] In accordance with yet another aspect of the invention, theportable computing device can rely on additional databases orinformation contained on a remote server computing device, or on aninfrastructure device located near the user. Thus, the portablecomputing device can reference the server computing device foradditional database information which is not stored locally, or toperform computationally intensive processes, such as calculating avisibility region. The portable computing device can also referenceinfrastructure devices to download a more localized database, to learnthe location of the infrastructure device with which it iscommunicating, or to learn the location last determined by similarportable computing devices located within the user's vicinity.

[0034] In keeping with the invention, a database is referenced by aportable computing device to determine a user's location based upon theuser's input describing the visual scene around them. Turning to FIG. 2,an exemplary environment is shown, with a user 200 and variouslandmarks, including buildings 202, 204, and 206, a water tower 208, anda radio tower 210. As an initial matter the exemplary environment shownin FIG. 2 will be used to describe databases contemplated by the presentinvention. In addition, the exemplary environment shown in FIG. 2 willalso be used to illustrate the determination of the location of theuser. As will be understood by those skilled in the art, the two stepsof acquiring the database and using the database take place at differenttimes and are not required to be done consecutively.

[0035] One database contemplated by the present invention is a simpletextual database correlating landmarks to common descriptive terms auser might use to describe them. For example, building 202 is shown as ablack skyscraper. The user is likely to comment on the color of thebuilding, and might refer to it as “black”, or “dark”, or perhaps“tinted”. Similarly, the user is likely to comment on the building'sheight, and might call it “tall”, “big”, “large”, or a “skyscraper”. Inaddition, there may be other features of the building that the usermight notice, such as “red antennas” or “white columns”. Each of thesedescriptions can be entered into a database together with the locationof the building 202. Such a location can be identified using a localizedCartesian grid, the position in latitude and longitude, such as thatprovided by a GPS receiver, the building's relationship to the nearestintersection, or any other location identifying format. A similarprocess can be performed on the other buildings 204 and 206, and on thewater tower 208 and the radio tower 210. Table 1 below illustrates anexemplary simple textual database correlating the buildings, theirlocation, and descriptive terms which would be commonly used to describethe corresponding building. TABLE 1 Size Color De- De- Building Heightscrip- scrip- Other ID Location Description tion tion Features 202 150,150 tall large black red antennas big dark white columns tinted 204 450,150 short large brown big tan 206 450, 450 medium medium red 1500 normalnormal brick sign average average pink 208 150, 450 tall tall white namemedium beige METROPOLIS normal light average 210  50, 450 tall tall redblinking thin white red light striped

[0036] For purposes of illustration, Table 1 contains a column for the“Building ID” which, in the present example, references the numberprovided in FIG. 2. However, the building ID can be any identifier thatcan be used to refer to the building across varying databases, ifnecessary, such as when calculating a visibility of the building, orwhen determining the location of the user. Similarly, while Table 1 onlycontains columns for “Height Description”, “Size Description”, and“Color Description”, any number of descriptive divisions can be made inthe database, or, as another alternative, each of the terms can bestored individually, rather than grouped by the property they describe.

[0037] The exemplary database of Table 1, above, also provides amechanism to store additional descriptive features of a building, suchas a prominent address, name, or other visually unique feature. Forexample, the red building 206 of FIG. 2 can contain a large addressnumber of 1500 displayed prominently on its facade. While the user maynot know what street they are on, and thus derive no locationalinformation from the sign, the portable computing device 20 can use theexistence of the sign as a mechanism by which to uniquely identify thebuilding based on the user's input. Similarly, the radio tower 210 canhave a blinking red light that may be described by a user and can,therefore, be stored in the table.

[0038] Additional information to that described above can be added tothe textual database, or stored in an alternative database, andreferenced by the portable computing device when attempting to determinethe user's location. Alternatively, the information contained in thetextual database above can be stored as additional data, or asmeta-data, within another database, such as the three-dimensionaltopographical database which will be described in detail below. As willbe appreciated by those of skill in the art, the textual database abovecan determine the user's location to a rough approximation. By takinginto consideration where the landmarks described by the user are visiblefrom, the location of the user can be determined with greater precision.

[0039] A three-dimensional topographical database can model the threedimensional properties of an area, providing the data from whichvisibility regions can be determined. Such a three-dimensionaltopographical database can be generated using a variety of techniques,including automated data gathering techniques, and techniques involvingmanual input. As an initial step, images of the region can be taken,either manually or through an automated process. Generally, overlappingimages providing a complete view around the point at which the imagesare being taken can provide an optimal input from which to construct thethree-dimensional topographical database. Such images can also begenerated from fish-eye lenses that can capture a surrounding panoramain a single photo. Alternatively, the photographs need not overlap, butcan provide a minimum of information necessary to view all of the sidesof the object to be added to the database.

[0040] Together with the images, the location of the camera can beprovided to aid in the generation of the three-dimensional topographicaldatabase and to provide information regarding the angles and distancescontained in the images. Based on the location of the camera, and theinformation within the images, a mosaic can be constructed to provide amore complete view. Such a mosaic can be simply the view around a givenpoint, or it can be a three dimensional view of an object from twodimensional images taken from various angles.

[0041] Once the image data has been entered, known algorithms can scanthe images and extrapolate from them the outlines of objects containedwithin the image. Again, such a process can be automated, or it can relyon human guidance. One method for inferring structure from the datacontained in the images is to use a feature-based algorithm. Afeature-based algorithm infers structure from the proximity of features,such as corners or edges. An alternative algorithm is a region-basedalgorithm that attempts to infer structure by identifying a largesection of an image that appears to be a surface, such as a wall ofbuilding. Yet another alternative algorithm is known as “edgehistogramming” and it seeks to identify a vertical facade by assumingthat such a facade would have many horizontal edges. Each of thesealgorithms can also be guided by human input, such as to identifyparticular walls that may be difficult for the algorithms to identify,or to guide the algorithms to a particular section of the image, or aparticular shade of color to aid in the identification of the buildingstructures. As will be appreciated by those skilled in the art, theabove algorithms are merely exemplary, as the present invention does notrely on a particular method for obtaining the three-dimensionaltopographical database.

[0042] Some of the above algorithms yield only geometric information,without color or texture, while others provide geometry with color andtexture. In order to determine visibility regions, the color and textureof a building are not of prime importance and can be sacrificed tomaintain a manageable size for the three-dimensional topographicaldatabase. Furthermore, color and texture information, to the extent thatthey would be noticed by a user and input into the system, could becontained more efficiently in the simple text-based database describedabove.

[0043] Because the geometry that is obtained by the algorithms describedabove can contain details that may not be required to determine avisibility region, such details can be omitted for additional storageand processing savings. For example, the geometry derived by the abovealgorithms can contain details such as ledges, engravings, smallantennas and other similar protrusions from the building that do notsignificantly impact the visibility calculations. Turning to FIG. 3, theenvironment of FIG. 2 is shown in a strict two dimensional perspective.The skyscraper 202 is shown with a small antenna 302 protruding from acorner of the skyscraper. The antenna 302 may be included in a geometricmodel derived from images of the skyscraper 202, but is not likely to bea significant factor in calculating visibility regions. For example, ascan be seen from FIG. 3, a user's line of sight 322 is not verydifferent from an estimated line of sight 312 which does not take intoaccount the antenna 302. Using either the line of sight 322 or 312, thesame buildings and landmarks are going to be visible to the user 200.

[0044] Consequently, such antennas and other small protrusions need notbe retained in the three-dimensional topographical database. By ignoringthe protrusions, each building can be modeled by as few as four polygonsdepicting the four sides of the building. Such a simplification canallow a large metropolitan area with as many as 1 million buildings tobe modeled in a three-dimensional topographical database containing asfew as 4 million polygons—a manageable amount even with average portablehardware.

[0045] However, some protrusions and other geometric features may benoticed by the user and even though they do not materially affectvisibility calculations, may need to be retained in some form. Suchinformation can be retained as an entry in the textual database, forexample in the same manner as described above with respect to noticeablefeatures, or such information can be added as additional data in thethree-dimensional topographical database, or in another database. Forexample, if a building has gargoyles along its top, it is not likelythat such carvings will materially affect visibility, and the buildingcan be modeled as a simple structure with polygonal walls. However, theuser may notice the gargoyles and describe the building as “havinggargoyles on top”. Such input can be particularly useful, as will beseen in more detail below, because the number of buildings having such afeature is likely to be small, providing a much smaller list ofcandidate landmarks being described by the user. Thus, information aboutsuch unique geometric features can be retained to allow the portablecomputing device the ability to narrow its search of landmarks matchingthe user's input if the user describes such features.

[0046] An additional consideration when modeling a building as havingsimple polygonal walls is the affect on visibility calculations byreducing the building's geometry to a few simple polygons. If theresultant inaccuracies cause the visibility calculations to determinethat a given landmark is visible when it is, in fact, not visible, theaccuracy of the location of the user is not likely to be affected.Returning to FIG. 3, the warehouse 204 contains a loading dock awning304 that may affect visibility calculations. In attempting to model thewarehouse 204 as a four polygons, and thereby removing the geometricinformation of the awning, the polygons that will represent the wallscan be placed either at the boundaries depicted by the warehouse 204 orby the warehouse 204 together with the awning 304. If the warehouse 204is modeled as four polygons placed at the edges of the warehouse 204,the user 200 will be determined to have a line of sight 314. However, ifthe warehouse 204 is modeled as four polygons placed at the edges of thewarehouse 204 with the awning 304, then the user 200 will be determinedto have a line of sight 324.

[0047] If the warehouse 204 is modeled in such a manner that the user'sline of sight is determined to be 314, then visibility calculations willconclude that the red building 206 is visible to the user. However, thered building 206 may, in fact, not be visible to the user because theuser's actual line of sight is line 324. In such a situation, thedetermination of the user's location is not likely to be affectedbecause the user cannot see building 206 and will, therefore, notdescribe it in the input provided to the portable computing device 20.Because the user's location is based on landmarks that are described bythe user as input into the portable computing device, the mere fact thatthe user does not describe building 206 should not affect thedetermination of the user's location.

[0048] However, it should be noted that the converse of the abovescenario may cause difficulty in determining the location of the user.If a simplified geometric representation of a building results invisibility calculations indicating that a given landmark is not visiblefrom a particular location when the landmark is, in fact, visible, thelocation of the user may be determined incorrectly. For example, in FIG.3, if the warehouse was modeled geometrically as four polygons placed atthe edge of the awning 304 and the warehouse 204, the user 200 would bedetermined to have a line of sight 324. With a line of sight 324, theuser 200, at the location shown in FIG. 3, would not be able to seebuilding 206. However, if the user 200 can, in fact, see building 206because the awning 304 does not actually block the view, the user maydescribe building 206 as input into the portable computing device. Theuser's location may, then, be improperly determined because only areasfrom which building 206 is visible will be considered as potentiallocations of the user. The user's actual location, as shown in FIG. 3,will not be among these potential locations, because it will bedetermined, due to the geometric modeling, that building 206 is notvisible from that location.

[0049] To the extent that modeling buildings as polygons removesgeometric details that do not significantly affect visibility, onepreferred embodiment locates the polygons at the locations of the wallsor other dominant vertical surfaces. Another preferred embodimentlocates the polygons a given percentage towards the center of thebuilding from wherever a wall of the building is determined to belocated. For example, as described above, geometric information can beextracted from images of a building or a scene. Such extracted geometricinformation can contain significant details, such as ledges, antennas,or the like, that can obscure the wall of a building. While knownalgorithms exist for extrapolating the locations of major elements of ascene, such a wall, from details contained therein, such algorithms maynot be perfectly accurate. Because, as shown above, modeling a buildingas a smaller structure than it actually is does not significantly affectthe determined location of the user, one preferred embodiment locatesthe polygons modeling the building a certain percentage, such as 10%,closer to the center of the building than the determined location of thebuilding walls. In such a manner, the modeled buildings will usually besmaller that the actual buildings and the calculations of the locationof the user can remain accurate despite inaccuracies in the algorithmsdesigned to extrapolate the location of building walls, and other majorelements, from geometric information containing many additionalgeometric details.

[0050] As described in detail above, the memory storage required for thethree-dimensional topographical database can be reduced by modeling thebuildings and other landmarks as geometric shapes represented bypolygonal sides. Another manner in which the memory storage requirementsof the three-dimensional topographical database can be reduced is todivide it into subsections, such as a city area, a particular region, orthe like. The subsections can be individually provided to the portablecomputing device 20, as needed, such that only a fraction of the wholethree-dimensional topographical database need be stored locally at anytime. Various methods can be used to deliver the sections of thethree-dimensional topographical database to the portable computingdevice 20 as needed by the user, including various client-serverconnections and reliance upon infrastructure.

[0051] One preferred embodiment for providing sections of thethree-dimensional topographical database to the portable computingdevice 20 is to enable a client-server link between the portable device20 and a three-dimensional topographical database server such as servercomputing device 49. The present invention contemplates various wirelessconnections to the three-dimensional topographical database server. Forexample, in a relatively small environment, such as the inside of a mallor office building, the portable computing device can remain incontinuous communication with a server through a wireless Ethernetprotocol. Alternatively, in larger environments, such as a city ordowntown area, the portable computing device 20 can wirelessly connectto the server computing device 49 through common cellular telephone andcellular data packet protocols. As will be seen by those skilled in theart, the present invention does not require the use of a particularprotocol or connection, though a faster connection can increase theefficiency of the communication, and may improve the user's experience.

[0052] Due to the significant amounts of memory storage that thethree-dimensional topographical database can require, and becausewireless communication generally does not have a high throughput, it maybe advantageous to transmit smaller sections of the topographicaldatabase more often, rather than to provide the user with largersegments that require a noticeable, or unacceptable, time to download.Additionally, the portable computing device is likely to have memorystorage so that additional sections, not currently used by the user, canremain in memory, decreasing the need to communicate with the server.For example, the portable computing device 20 can have, pre-installed, asignificant portion of the overall three-dimensional topographicaldatabase, or those segments of the three-dimensional topographicaldatabase most commonly used. In such a case, the portable computingdevice 20 would only need to communicate with the server and obtainadditional segments if the user traveled beyond the area modeled by thetopographical database installed on the portable computing device.

[0053] The simple text-based database described above can be used todetermine which segments of the three-dimensional topographical databaseshould be requested from the server computing device 49. The user canprovide input to the portable computing device describing variouslandmarks visible from the user's present location. Based on thetext-based database, the user's location can be approximated. If theportable computing device does not already have this segment of thethree-dimensional topographical database resident locally, it canrequest it from the server computing device 49. Alternatively, theportable computing device can track the movements of the user, anddetermine the appropriate segment to request from the server computingdevice based on the direction of travel of the user. The portablecomputing device can also have installed thereon additional locationsensors, such as a GPS receiver, and can track the user's movementsbased on the location as provided by the GPS receiver, when available,and based on previously determined locations using the landmarkidentification of the present invention. In addition to requestingsegments of the three-dimensional topographical database as needed, theportable computing device can request them in advance and buffer themuntil needed. One method for efficiently requesting the segments inadvance is to estimate the direction of the user's travel, as describedabove, and extrapolate the direction beyond the next segment. In such amanner, if the user continues in their present course, the next segmentwill already be buffered on the portable computing device 20. Using sucha system it is possible to continually buffer new segments before theyare needed to determine the user's location, resulting in moreresponsive location determination.

[0054] Another preferred embodiment of the present invention avoids theneed to maintain or initiate a client-server link by transferringadditional segments of the three-dimensional topographical database tothe portable computing device 20 through infrastructure devices such asinfrastructure device 62. One such infrastructure device can be a kioskat which the user can aim the portable computing device 20 and receive asegment of the three-dimensional topographical database through aninfrared or radio frequency wireless connection. Because kiosks can beplaced in known locations, the segment of the three-dimensionaltopographical database that is provided by the kiosk can be the segmentof the area surrounding the kiosk. An alternative infrastructure devicecan be a base station that broadcasts a segment of the three-dimensionaltopographical database. Again, because the base station can be placed ina known location, and the broadcast range can be limited compared to thecoverage of the segments of the topographical database, the base stationcan broadcast only the segment of the topographical database coveringthe vicinity of the base station.

[0055] By using infrastructure device 62 broadcasting a given segment ofthe three-dimensional topographical database, the portable computingdevice 20 does not need to request a particular segment. Additionally,because these infrastructure devices may have a small broadcast range,their location can simultaneously be the location of the user when theportable computing device 20 is close enough to receive thecommunication from the infrastructure device 63. The portable computingdevice 20 can then pass the location provided by the infrastructuredevice 63 onto the user, or can use it as a starting point from which todetermine the location of the user as the user moves beyond the range ofthe infrastructure device.

[0056] Turning to FIG. 4, a flowchart is shown illustrating onepreferred method for using the portable computing device described aboveto determine the user's location. Initially, at step 402, the portablecomputing device receives a user input describing landmarks and othervisual references visible to the user at the user's present location. Asdescribed above the portable computing device 20 can include variousinput devices, such as a keyboard 40 or a microphone 42. Because theportable computing device is likely to be carried by the user, andlikely to be quite small so as not to unduly burden the user, onepreferred embodiment relies on speech input through the microphone 42 toreceive the user's input. Alternatively, larger portable computingdevices can use keyboard, mouse, or various other input devices toreceive the user's input. As is known by those of skill in the art,numerous speech recognition techniques are available for obtainingcomputer-readable input from a user's vocal input. The present inventionis not limited to the use of any particular technique for interpretingvocal input, though it is important to note that complex algorithms maynot be necessary because it is likely that the user's input will containa limited vocabulary and not be continuous speech. Thus, the vocalinterpretation algorithms selected can remain efficient because the useris likely to use only descriptive terms and architectural nouns incommunicating the landmarks currently visible to the user. Similarly,the vocal interpretation algorithms selected need not be concerned withinterpreting continuous, conversational speech, as the user is likely touse short, staccato phrases and terms to describe the currently visiblelandmarks.

[0057] The user input of step 402 can be a description of a singlelandmark, or of all the landmarks visible to the user at the presentlocation. If the user does not provide sufficient information at step402, the portable computing device 20 can request additional informationat step 414 or 416, as will be described further below. Returning to theuser input at step 402, it is expected that the user will providesufficient information to enable the portable computing device 20 todetermine the user's position. Such input can include specificcharacteristics of the landmarks visible to the user. For example, theuser can identify the color of a landmark, its approximate height, itssize, and any other distinguishing features, such as signs, colors,lights, carvings, or other distinctive visual features. In general, itis more helpful if the user provides specific descriptions as it allowsthe portable computing device 20 to more quickly and more accuratelydetermine the landmark the user is describing. For determining theuser's location, it is also helpful if the user provides the relativelocation of the landmark the user is describing, such as whether it isin front or behind the user, or whether it is to the user's left or tothe right. Such information can be used further aid in the determinationof the user's input, such as determining whether the user is between twolandmarks.

[0058] To ensure appropriate user input, the portable computing device20 can interact with the user through either visual output, such asthrough a screen or other display, or through audio output, such asthrough a speaker or earphones. If portability is of primary concern,audio interaction may be preferred because the size of the portablecomputing device 20 can be significantly reduced if it does not need anydisplay capabilities. Using the feedback to the user, the portablecomputing device can request or guide the user's input. For example, theportable computing device can request that the user identify specificcharacteristics of the landmarks the user sees. The portable computingdevice 20 can also guide the user to help the user select the landmarksto describe. For example, the portable computing device can request thatthe user describe landmarks close to the user, rather than largerlandmarks that are further away. As will be understood by those skilledin the art, a large landmark has a large visibility radius and does notaid determining the user's location. Additionally, the user can bedirected to provide partial input regarding landmarks, even if the userdoes not have more complete information. As will be explained furtherbelow, the user's location is based on as many inputs as the user canprovide, and a landmark need not be identified completely. Thus, if theuser sees a red building, but cannot determine its size, the mere factthat the user is near a red building can aid the portable computingdevice 20 in determining the location of the user. However, many usersmay not understand this and, thus, withhold partial information becausethey cannot provide more complete information. The feedback from theportable computing device can guide the user and request this input.

[0059] Another alternative contemplated by the present invention is fora video or image capture device to provide an image as a component ofthe user's input at step 402. The image could be compared to a series ofstored images, or it could be used in a manner similar to that describedabove and have information regarding the user's surroundingsautomatically extracted from it through feature recognition algorithms.The image capture device can be part of the portable computing device20, or a separate attachment connected through common communicationprotocols, such as USB or Firewire. Additionally, the image capturedevice can automatically capture images or can do so under the controlof the user. For example, the portable computing device 20 couldautomatically activate the image capture device, or request that theuser capture the image of a particular structure, to obtain furtherinformation if the user's input was not sufficiently descriptive.

[0060] Returning to FIG. 4, at step 404, the portable computing device20 can determine the landmarks that the user has identified as beingvisible from the user's current location. In a preferred embodiment, allof the user's input need not be matched to a specific landmark availablein the databases, but need only be matched to a list of candidatelandmarks. Thus, if the only input provided by a user was theidentification of a building as red, the list of candidate landmarks cancontain all of the red buildings in the databases. Such a list ofcandidate landmarks can be created for all of the inputs provided by theuser.

[0061] The list of candidate landmarks can then be further trimmed byconsidering the visibility region of each landmark, at step 406. As willbe recognized by those of skill in the art, the determination of thevisibility region of a landmark can be a computationally intensiveprocess. As will also be recognized by those of skill in the art, manyalgorithms exist for such computation and the present invention is notintended to be limited to any particular algorithm, although moreefficient algorithms are likely to be more suitable for the computingabilities of a common portable computing device, and are likely toresult in a more satisfactory user experience.

[0062] Generally, the calculation of a visibility region comprisescalculating a line between a visibility point in space and a point onthe building and then determining whether that line intersects any otherobjects. If the line crosses any other objects, then the selected pointof the building is not visible from the visibility point in space.Optimizations can be achieved by limiting the points selected on thebuilding to be only those points at the very top, or that wouldotherwise provide the most visibility. Additional optimizations can beachieved by limiting the area in which the visibility point can beselected. For example, most people are between 4 and 6 feet tall, andthe visibility point can be selected at a common height within thisrange. Alternatively, the portable computing device 20 can provide for auser input specifying the user's height, and always select visibilitypoints at that height. Visibility points can also be selected so as notto coincide with areas where an object already exists, and would preventthe user from standing there. By eliminating such options, the range ofvisibility points and points on the building narrows, resulting in amore efficient visibility region calculation.

[0063] In another preferred embodiment, step 406 can be performed inpart by the portable computing device 20, and in part by the servercomputing device 49. The server computing device 49 can calculate,either in advance, or as needed, the visibility regions for commonlandmarks that may be often described by a user. The portable computingdevice 20 can, therefore, rely on the greater processing power of theserver computing device 49 and simply request the visibility region forany common landmarks necessary to determine the location of the user. Byonly computing the visibility regions for common landmarks, the servercomputing device 49 can minimize the amount of time spent on specializedrequests, and can more efficiently serve a greater number of portablecomputing devices. As an alternative, however, the server computingdevice can calculate visibility regions on request, such that theportable computing device can send a list of candidate landmarks and theserver computing device can return their visibility areas, or even theintersection of their visibility areas. Because of the increasedprocessing power of the server computing device 49, the location of theuser can be determined more efficiently if the portable computing device20 relies, at least in part, on calculations performed by the servercomputing device.

[0064] At step 408, the portable computing device 20 can calculate theintersection of the visibility regions determined at step 406. Theintersection of a visibility region is the location where bothlandmarks, whose visibility regions are being intersected, are visible.Thus, the intersection of visibility regions is the area covered by bothvisibility regions. As more visibility regions are intersected with oneanother, the area of their intersection generally tends to decrease. Bydefinition, the user must be located at a position where all of thelandmarks the user described are visible. Thus, the intersection of thevisibility regions of all of the landmarks described by the user willidentify the user's location, or a small area containing the user'slocation.

[0065] Step 408 also aids in trimming the candidate landmarks andidentifying the landmark described by the user. Many of the candidatelandmarks determined at step 404 may have no intersecting visibilityregions, and can, therefore, be eliminated. For example, returning toFIG. 2, the user 200 may describe a red building 206, a brown warehouse204, and a white water tower 208. There may be a number of redbuildings, brown warehouses, and white water towers in the databasesavailable to the portable computing device 20. However, it is likelythat very few of them will have a visibility region that intersects withother visibility regions to provide an area, or a point, from which onecould see all three at the same time. Those candidate red buildings,brown warehouses, and white water towers whose visibility regions do notintersect can be discarded as candidates. Additional candidates can bediscarded based on the relative location of the landmarks provided bythe user. For example, if the user specifies that one landmark is infront of the user, and another is behind the user, the area in which theuser can be located can be narrowed. Some candidate landmarks, then, maynot be visible from this narrowed area, and can be discarded. Whensufficient candidates have been discarded, the intersection of theresulting candidates' visibility regions defines the area in which theuser is currently located. Additionally, the relative location inputform the user can be used to further narrow the area in which the useris currently located. Depending on the settings at step 410, to bedescribed further below, the portable computing device may havecompleted its task of locating the user, or it may need additional inputto do so to a sufficient level of accuracy.

[0066] Candidate landmarks whose visibility regions do not coincide withthe visibility regions of other candidate landmarks described by theuser can be discarded even if the discarded candidate landmark was theonly candidate landmark for a description provided by the user. Bydiscarding candidate landmarks, even if the result is the exclusion of acomponent of the user's input, the system can maintain robustness in theface of an inaccurate description by the user, an inaccurate entry inone of the databases, or an improper candidate landmark determination atstep 404. For example, if the user inadvertently described water tower208 of FIG. 2 as a yellow water tower, the step 404 may find a candidateyellow water tower located far from the other landmarks the userdescribed, including building 206 and warehouse 204. Additional inputfrom the user regarding the specific landmarks described may not be ableto solve the fundamental problem that the only yellow water tower in thedatabases is nowhere near the other landmarks described by the user, andthus there is no location at which a yellow water tower is visibletogether with building 206 and warehouse 204. In such a situation, thelocation of the user 200 can be accurately determined by discarding theuser's input of a yellow water tower. A similar scenario can result ifthe databases contain an incorrect location for the water tower 210.

[0067] While the embodiment of FIG. 4 shows steps 406 and 408 asindependent and occurring in a particular temporal order, anotherembodiment can combine steps 406 and 408 in such a manner that step 406need not be completed prior to commencing step 408. Such an arrangementcan result in efficiencies because visibility regions arecomputationally expensive to calculate. Thus, step 406 can initiallydetermine the visibility regions of the landmark candidates from theuser descriptions that resulted in the fewest candidate landmarks. Thosevisibility regions can then be intersected to yield a smaller region atstep 408. By first determining the visibility regions of those landmarksdescribed by the user that resulted in the fewest candidate landmarks,the portable computing device 20 can take advantage of the decreasingvisibility intersection area by determining whether other candidatelandmarks are visible from the intersection area, instead of calculatingtheir entire visibility area and intersecting it. Such a technique isanalogous to query optimization in database systems. For example,returning to the example from FIG. 2 above, if the user described thered building 206, the brown warehouse 204, and the white water tower208, step 406 may determine the visibility region of the white watertower first because there may only be two such water towers in thedatabases. After the visibility regions of the two white water towerswere determined, they could be intersected with the visibility regionsof the brown warehouses contained in the databases, of which there maybe only five. The only intersection in these visibility regions willlikely be the intersection of the visibility region of water tower 208and warehouse 204. As a result, the portable computing device 20 doesnot need to determine the visibility regions of all of the red buildingsin the databases, of which there could be quite a few. Instead, it issufficient to determine the visibility of the candidate red buildingsonly from the area defined by the intersection of the visibility regionof water tower 208 and warehouse 204. Whichever building can be seenfrom that area is likely the building identified by the user. Then itswhole visibility region can be determined and intersected withintersection of the visibility region of water tower 208 and warehouse204, yielding an even smaller area, and a more precise determination ofthe user's location. By avoiding the calculation of the visibility areasand intersections of those landmarks described by the user that resultedin the most candidate landmarks, the portable computing device canachieve a greater efficiency in determining the location of the user.

[0068] Another preferred embodiment uses a hierarchical approach tosteps 406 and 408 shown in FIG. 4. To determine the visibility region ofa candidate landmark at step 406, the area covered by the database canbe divided into segments, and each segment can be tested to determinewhether the candidate landmark may be visible from that segment. Thecandidate landmark may be visible from a segment if there is at leastone point in that segment from which the candidate landmark is visible.Those segments from which the candidate landmark is not visible do notneed to be considered in further visibility calculations. At step 408,the segments from which a first candidate landmark is visible can beintersected with the segments from which a second candidate landmark isvisible by simply finding the segments from which they are both visible.By operating at a segment level, as opposed to an individual pointlevel, the hierarchical approach allows greater efficiencies in theinitial calculations. Once a sufficient number of candidate landmarks'visibility regions are intersected, there can be few resulting segmentsfrom which all of these candidate landmarks are visible. Then, returningto step 406, the visibility region can be determined for each candidatelandmark testing every point in the segment, but only for the segmentsfrom which all of the candidate landmarks were determined to be visible.These more accurate visibility regions can then be intersected at step408 to determine the user's location. By reducing the size of the areafor which the more accurate visibility region can be calculated andintersected, the hierarchical approach can result in greaterefficiencies.

[0069] Ultimately, the visibility calculations of the candidatelandmarks at step 406 and their resulting intersection at step 408 arelikely to result in a small area, or possibly a unique point, from whichall of the landmarks described by the user are visible. By definition,then, this is the user's location, and the portable computing device 20can inform the user of their location at step 412. Prior to informingthe user, the portable computing device can determine its level ofconfidence regarding the derived location of the user at step 410exceeds a threshold confidence level. The threshold confidence level canbe dependent on a predetermined threshold confidence area can beprovided to the portable computing device 20. For example, if thethreshold confidence area is a city block, or a few feet in an indoorenvironment, and the derived location of the user is accurate to withina city block or less, or accurate to within a few feet or less in anindoor environment, the user can be presented with the derived location.However, if the portable computing device 20 ultimately determines thatthe landmarks described by the user are all visible from an area largerthan the threshold confidence area, such as an area encompassing manycity blocks, the computing device can perform additional steps prior toproviding the determined location to the user. Alternatively, thethreshold confidence level can be dependent on a user defined thresholdconfidence area that can be set by a user. Thus, if the user onlyrequires an approximate location, a more lenient threshold area can beused by the portable computing device 20 and, as a result, the user willbe able to obtain their location more efficiently, as the computingdevice will sooner determine a visibility region within the more lenientthreshold. Conversely, a user who sets a more stringent threshold mayfind that the portable computing device requires more information beforeit can provide a determined location that meets the threshold confidencelevel.

[0070] In another preferred embodiment, the determination of thevisibility regions and their intersections at steps 406 and 408 can takeinto account a decreasing probability of visibility as the distance fromthe selected visibility point to the landmark increases. Because of thesimplification process described above, wherein the three-dimensionaltopographical database models its environment as comprised of simplepolygonal walls, and because many image recognition techniques do notextrapolate details such as trees, leaves, telephone lines, and thelike, the present invention can take such additionalvisibility-impacting elements into account by reducing the percentagechance that the landmark is visible from the visibility point as thedistance between the visibility point and the landmark increases. Thus,even though the visibility calculations indicate that there is noobstruction between the selected visibility point and the landmark, theportable computing device can take into account various other factorsaffecting visibility and decrease the percentage chance that thelandmark is, in fact, visible from the selected visibility point, as thevisibility point is moved further away. For example, even thoughvisibility calculations at step 406 may indicate that the landmark isvisible from a selected visibility point half a mile away, the portablecomputing device can use a visibility probability of only 50% for thatvisibility point. Such a model can more closely approximate reality asthere is a less than 100% certainty that a user can see a particularlandmark half a mile away in a crowded city, even if there is no wall inthe user's line of sight, because other obstructions, not currentlymodeled in the three-dimensional topographical model, may interfere withthe user's visibility. As will be understood by those skilled in theart, the percentage visibility can decrease as the distance from thelandmark increases in a manner proportional to the inverse of the squareof the distance from the landmark. As will also be understood by thoseskilled in the art, the intersection of the visibility regions at step408 can be determined by multiplying the visibility percentages of theindividual visibility regions being intersected.

[0071] If steps 406 and 408 determine the visibility probability, theconfidence, as determined at step 410, can also take into account thevisibility probability, in addition to the size of the area containingthe user's location. The area containing the user's location can bedetermined by the intersections performed by step 408 as described indetail above, and can be expressed as an area with decreasingprobabilities of visibility of all of the landmarks described by theuser as the distance from a central area increases. Such visibilityprobabilities can also indicate the probability that the user is in thedetermined area. Thus, there may exist a central area, or a singlepoint, at which all of the landmarks described by the user are visibleand their visibility is 100%. Further from this area, the landmarks maystill all be visible, but the visibility probability decreases. Onemethod for taking into account the visibility probability whendetermining the confidence of the user's location can be by comparingthe area determined to contain the user's location at a given visibilityprobability to the threshold confidence area. As described in detailabove, the confidence area can be pre-determined, or it can be selectedby the user based on the user's needs. Similarly, a thresholdprobability can also be pre-determined or it can be selected by the userbased on the user's needs. Once the threshold probability is determined,the area determined by step 408 at the threshold probability can then becompared to the threshold confidence area at step 410 to determinewhether there is sufficient confidence in the result of the computationsthrough step 408. For example, if the threshold probability is selectedto be 50%, and the threshold confidence area is selected be one cityblock, then the user's location is confidently determined when the areadetermined by step 408 is such that all of the landmarks described bythe user are visible, with a 50% probability, or higher, from an areaone city block, or smaller. By including the visibility probability inthe calculations, the user can adjust the precision of the locationdetermination using two variables: the visibility probability and theconfidence area.

[0072] As shown in step 414, if, at step 410, it is determined that athreshold confidence level has not been reached, the portable computingdevice 20 can ask that the user provide additional input. The user canprovide additional input regarding the landmarks the user has alreadydescribed, which can reduce the number of candidate landmarks, or theuser can describe a new landmark, which can provide another visibilityarea that can decrease the area from which all of the landmarksdescribed by the user are visible. Alternatively, as shown in step 416,the portable computing device 20 can request specific information fromthe user. Thus, if a landmark description by the user creates a largelist of candidate landmarks at step 404, but only a few, or only one, ofthe candidate landmarks contain a unique feature, the portable computingdevice can determine the unique feature and ask the user whether thelandmark the user is describing contains this feature. Such aquestion-and-answer feedback cycle can continue until the list ofcandidate landmarks for each of the landmarks described by the user hasbeen reduced sufficiently such that the results of step 408 have asufficient confidence level at step 410. The portable computing device20 can also ask specific questions of the user when determining the areawhere each of the landmarks described by the user can be seen. Forexample, returning to FIG. 2, if the computing device 20 has calculatedtwo areas from which a user could see the warehouse 204, the building206, and the water tower 208, but only one of those areas also providesvisibility to the radio tower 210, the portable computing device can askthe user whether the user sees a radio tower. User 200, shown in FIG. 2,may respond in the negative, since the user's view of the radio tower islikely obstructed by the skyscraper 202. By providing that answer, theuser can allow the portable computing device to more accuratelydetermine the user's position, and have sufficient confidence at step410.

[0073] While FIG. 4 illustrates steps 416 and 414 as the result of theconfidence analysis at step 410, another preferred embodiment canperform step 414, step 416, or both after either step 404 or step 406.For example, step 416 can be used to narrow the list of candidatelandmarks determined at step 404 prior to any visibility calculations atstep 406, resulting in an increased efficiency due to the avoidance ofthe computationally expensive visibility calculations. Similarly, step414 can be used to identify a particular visibility region if step 408results in two or more non-contiguous visibility regions. The presentinvention contemplates feedback to the user, in the form of questions,whenever appropriate to create efficiencies or provide a more accuratelocation to the user.

[0074] Finally, as shown in step 412, once an appropriate level ofconfidence is reached, the user can be informed of their location. Ascan be seen, the present invention provides mechanisms by which a user'slocation can be determined through a description of the user'ssurroundings. The present invention does not require a significantinfrastructure, nor does it impact the user's privacy. Instead thepresent invention relies on the same elements that users have relied onfor many years to provide location and direction information: commonlandmarks and other visual cues.

[0075] Though the above description has generally referred to theportable computing device 20 as performing certain actions, it is notlimited to implementation only on a portable computing device. Onealternative contemplated by the present invention is that the portablecomputing device 20 can act as a thin client and simply receive userinput and pass the user input on to a server computing device 49. Theserver computing device 49 can then perform the above functionsdescribed as performed by the portable computing device. For example,the server computing device 49 can utilize its increased processingresources to more efficiently determine candidate landmarks based on theuser's input, calculate visibility regions for those landmarks, anddetermine an intersection of those landmarks to derive the location ofthe user, in the manner described in detail above. Once the location ofthe user is derived, or if the server computing device 49 requiresadditional input, it can contact the portable computing device 20 torequest the additional input from the user, or to provide the derivedlocation to the user.

[0076] Another alternative contemplated by the present invention is theuse of infrastructure communication between portable computing devicescan be used, in addition to user input, to determine a user's location.Thus, one portable computing device can listen for, or seek out, otherportable computing devices in its immediate vicinity. If such otherportable computing device are in the vicinity, the portable computingdevices can poll each other to determine the most recent derivation of auser's location and, assuming the derivation was sufficiently recently,can use the user's location as derived by another portable computingmachine, either as another input, or as the final result presented tothe user.

[0077] Yet another alternative contemplated by the present invention isthat the portable computing device can act in conjunction with, or as aback-up to, other known locating mechanisms, such as GPS or E-911. Forexample, the present invention can be used by a GPS receiver to providethe user's location in a downtown area where satellite reception isdifficult and unreliable. Alternatively, the GPS receiver could use thepresent invention to verify the location it derives based on thesatellite signals. The present invention also contemplates the converse:the known locating mechanisms can act in conjunction with, or as aback-up to, the present invention. Thus, a GPS receiver can be installedonto portable computing device 20 to verify the derived location of theuser. For example, the known locating mechanisms can provide additionalinput with which to determine the confidence of the derived userlocation at step 410.

[0078] In view of the many possible embodiments to which the principlesof this invention may be applied, it should be recognized that theembodiment described herein with respect to the Figures is meant to beillustrative only and should not be taken as limiting the scope ofinvention. For example, those of skill in the art will recognize thatthe elements of the illustrated embodiment shown in software may beimplemented in hardware and vice versa or that the illustratedembodiment can be modified in arrangement and detail without departingfrom the spirit of the invention. Therefore, the invention as describedherein contemplates all such embodiments as may come within the scope ofthe following claims and equivalents thereof.

[0079] All of the references cited herein, including patents, patentapplications, and publications, are hereby incorporated in theirentireties by reference.

1-45. (Cancelled).
 46. A database of landmarks comprising, for eachlandmark in the database: topological information related to thelandmark, wherein the topological information describes a larger andmore geometrically simple structure than the landmark; anduser-noticeable features related to the landmark, wherein theuser-noticeable features are prominent features or features thatdistinguish the landmark from similar landmarks in the database.
 47. Thedatabase of claim 46, wherein the topological information isautomatically derived from a series of overlapping images.
 48. Thedatabase of claim 47, wherein the topological information isautomatically derived from the series of overlapping images using edgehistogramming.
 49. The database of claim 46, wherein the user-noticeablefeatures are added to the database via manual annotation.
 50. Thedatabase of claim 46, wherein the user-noticeable features are describedin the database with common linguistic terms used to describe theuser-noticeable features.
 51. The database of claim 46, wherein eachlandmark in the database is either likely to be identified by a user oris large enough to cause occlusion of at least one other landmark in thedatabase.
 52. A method for generating a database of landmarkscomprising: deriving topological information from a series ofoverlapping images; quantizing the topological information to representa simple geometric structure; increasing the size of the simplegeometric structure to account for quantizing errors; generatingdescriptions of user-noticeable features of the landmarks; and linkingthe descriptions of the user-noticeable features to a correspondingquantized topological information.
 53. The method of claim 52, whereinthe simple geometric structure is only four polygons.
 54. The method ofclaim 53, wherein the deriving the topological information from theseries of overlapping images uses edge historgramming.
 55. The method ofclaim 53, wherein the user-noticeable features are prominent features orfeatures that distinguish a landmark from similar landmarks in thedatabase.
 56. The method of claim 53, wherein the user-noticeablefeatures are manually selected.
 57. The method of claim 53, wherein thegenerating descriptions of the user-noticeable features of the landmarksuses common linguistic terms used to describe the user-noticeablefeatures.
 58. The method of claim 53 further comprising removing eachlandmark from the database that is not either: (1) likely to beidentified by a user, or (2) is large enough to cause occlusion of atleast one other landmark in the database.
 59. A computer-readable mediumhaving computer-executable instructions for generating a database oflandmarks, the computer-executable instructions performing stepscomprising: deriving topological information from a series ofoverlapping images; quantizing the topological information to representa simple geometric structure; increasing the size of the simplegeometric structure to account for quantizing errors; generatingdescriptions of user-noticeable features of the landmarks; and linkingthe descriptions of the user-noticeable features to a correspondingquantized topological information.
 60. The computer-readable medium ofclaim 59, wherein the simple geometric structure is only four polygons.61. The computer-readable medium of claim 59, wherein thecomputer-executable instructions for deriving the topologicalinformation from the series of overlapping images implement edgehistorgramming.
 62. The computer-readable medium of claim 59, whereinthe user-noticeable features are prominent features or features thatdistinguish a landmark from similar landmarks in the database.
 63. Thecomputer-readable medium of claim 59, wherein the user-noticeablefeatures are manually selected.
 64. The computer-readable medium ofclaim 59, wherein the computer-executable instructions for generatingdescriptions of the user-noticeable features of the landmarks utlizecommon linguistic terms used to describe the user-noticeable features.65. The computer-readable medium of claim 59 further comprising removingeach landmark from the database that is not either: (1) likely to beidentified by a user, or (2) is large enough to cause occlusion of atleast one other landmark in the database.